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This study determined psychometric properties of 3-ET, an instrument 
specifically designed to track the effects of a curriculum support intervention 
created to enhance the delivery of the national curriculum in a war zone in 
Uganda. The instrument was developed through brain storm and expert 
review before being committed to structural and reliability testing using 
Exploratory Factor analysis (EFA). The 88 variables of 3-ET were reduced 
to 73; five distinct sub-scales emerged. Sub-scale 1, measured core education 
support functions and services. Sub-scale 2, assessed aesthetics/efficiency. 
Sub-scale 3, focused on barriers to stakeholder participation. Sub-scale 4 
measured peacefulness and gender sensitivity of learning environments. 
Sub-scale 5 assessed pupil participation. 3-ET has acceptable structural 
validity and internal consistence reliability. It can be used to test 
effectiveness of the REPLICA intervention. Its applicability in other conflict 
and non-conflict contexts needs review. 
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1. INTRODUCTION 

In 2005/6, the Ministry of Education and Sports introduced a new curriculum support intervention 
branded REPLICA-for revitalizing education, participation and learning in conflict areas. The intervention 
entailed introduction of peace education, psychosocial support, community integration, leadership and 
governance training, promotion of girls’ education and use of performing arts in learning into the formal 
teaching and learning program of the affected schools and school communities. Field anecdotes and official 
statistics had painted a grim picture of education performance in the war zone specifically because of the war 
related psychosocial and physical problems. The region had low net intake (NIR), completion and pass rates 
of 25-50%, 23-35%, and 1-24% respectively [1] and wide spread Psycho-trauma [2]. 

The learning centres that had been created within the internally displaced camps to ensure safety and 
better administration of schools had instead faced several challenges that directly constrained teaching, 
learning and stakeholder participation in education and ultimately the educational competitiveness of the 
entire region. The main challenges included overcrowding, filthy learning environments, entrenched culture 
of violence and administrative problems. Others were trauma, apathy and uncertainty among teachers and 
learners [3]-[4], over-stretched and degraded facilities; apathetic, unsupportive and dysfunctional school- 
communities, and traumatised human resources. Moreover the school-community interface had been 
distressed by war instigated suspicion and mistrust. Government was concerned about the poor academic 
performance, the lack of trained staff and predictability in the war zone. The REPLICA program was 
therefore developed as a curriculum support material. It was modelled around the recommendations of the 
International Network for Education in Emergency (INEE) which had recommended among others the 
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prioritization of community participation and the use of local resources; partnerships and inter-sectorial 
linkages; effective teaching and learning through curriculum, training, instruction and assessment reforms; 
administration and management of human resources, and policy formulation, planning, implementation, and 
coordination in the delivery of education services in such settings [5]-[6]. 

It was postulated that REPLICA would lead to significant improvements in school level 
administration; aesthetics, stakeholder participation; peacefulness & gender sensitivity of schools, and learner 
participation in co-curricular activities consequent upon exposure to the intervention ultimately translating 
into better Net Intake, Completion and Pass Rates. In order to track the hypothesized effects and assess 
program effectiveness, the 3-FT tool (ie Evaluating Education in Emergency Tool) was therefore created 
through a multi-stage and participatory process. The instrument was piloted in 30 none project primary 
schools before being deployed in the 681 of the 1,800 project schools. The 681 schools had been sampled to 
participate in an inbuilt impact tracking study. Data for 270 of the 681 schools were randomly extracted 
using STATA release 11 for purposes of the current study. 

Exploratory Factor analysis (EFA) was used to reduce the 88 variables of the original 3-ET to a 
smaller number of constructs [7] (Thompson et al., 1996): this approach has been used in other validation 
studies [8]-[ 14]. To Kerlinger (1979), EFA has been shown to be one of the most powerful methods for 
reducing variable complexity to greater simplicity [15]. McLeod et al., (1977) argues that the emergent 
factors may actually be used as variables in further analysis [16]. Kaiser criterion [ 17]-[ 18] and confirmatory 
scree plots [19] were used to select the factors and rotated factor load thresholds of 0.30, was applied as basis 
for item retention. Chronbach’s alpha coefficient of 0.70 was adopted as threshold for internal consistence 
reliability acceptance. Interpretation of Pearson r coefficients was based on Cohen’s recommendation [19] 
that coefficients of 0.10-0.29 are small, 0.30-0.49 are medium and 0.50-1 are large. STAT release 11 was 
used for data analysis. This paper discusses findings of the structural validity and internal consistency 
reliability analysis. 


2. RESEARCH METHOD 

2.1. Setting 

The original data were collected from selected schools in the Northern and North Eastern regions of 
Uganda. This region had had war since 1986 and all the schools were affected. The adverse effects of the war 
on the local educational outcomes had motivated the introduction of the REPLICA program by the Ugandan 
education ministry. The primary data were collected from the primary schools-each of the 13 participating 
districts was represented. All the social services in the targeted region had been destroyed during the war and 
a large proportion of the population had been relegated into internal displacement camps. The data for the 
current study were collected between January and March 2007. 

2.2. Design 

This study employed a cross-sectional design. 

2.3. Sample and sampling strategy 

The original data were collected from 681 schools. The school selection had been done by the 
Ministry of Education and Sports (MoES) for purposes of tracking intervention impact. The process of 
selection was multi-staged and blinded. District Education Officials, in consultation with the Core Primary 
Teacher Colleges, had submitted lists of schools to the Education Ministry in response to a pre-defined 
inclusion and exclusion criteria that had been developed by the project staff. These primary lists of schools 
were then used by the MoES to draw a stratified random sample of 698 schools. The current study employs 
records of a random sample of 270 of the 681 schools. 

2.4. Statistical analysis 

The assessed psychometric properties of 3-ET were: (a) internal factorial structure i.e. the constructs 
underlying the wide-array of REPLICA program indicators and the extent to which they measure the specific 
dimensions of education revitalization. Factor retention was based on Kaiser Criterion, and confirmed 
through scree plots since Kaiser Criterion tends to overestimate the number of reliable factors. Varimax 
rotation was performed to determine structure parsimony, (b) Internal consistency reliability i.e. internal 
consistency of items in the emergent factors, were tested using Cronbach’s Alpha. Variables with rotated 
factor loadings <0.30 were removed; in each case, reliability tests were run. This series of factor analyses and 
reliability tests continued until all variables had at least one >0.30 rotated factor loading and reliability 
coefficients stabilized: this approach has been widely used in the literature [7],[10]-[12]. To test whether 
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sub-scales measured distinct aspects of Educational revitalization, individual scale scores were summed and 
their relatedness were assessed using Pearson correlation tests. Non-significant correlations were considered 
indications of uniqueness. 


3. RESULTS AND ANALYSIS 

3.1 Sample characteristics 

Data from a random sample of 270 primary schools from Northern and eastern Uganda were 
analysed. 

3.2 Psychometric properties of assessment tool 

The 88 variables of 3-ET, were reduced through exploratory factor analysis to 73. The [15] omitted 
variables were: signpost covered by grass or bush; Display of School map highlighting surrounding barriers 
to girls’ education; Record of solutions or ways of minimizing the impact of above barriers; Latrines with 
lockable doors or an enclosure; Comments on schemes of work and lesson plan from Head teacher; Endorsed 
Teachers’ attendance book; Vision of the school visibly displayed; Teaching and learning going on at time of 
visit; P.l teaching & learning done in Local Language; Clear path leading to the school; Each class has a 
class teacher; Existence of sanitation prefect; Evidence of children’s active participation in class; Schedule 
for cleaning the school; and Movable school signpost. 

The retained variables were clustered in five distinct sub-scales (as 1 summarized in table 1). Sub¬ 
scale (factor 1), with 20 items had rotated factor loads ranging between 0.31162 and 0.75402, with internal 
consistency reliability coefficient of 0.8788. It measured core education support functions. Sub-scale (factor) 
2, assessed aesthetics/efficiency of service delivery. It had 21 items with rotated factor loads ranging between 
0.37233 and 0.81262, and internal consistency reliability coefficient of 0.8821. Sub-scale 3, focused on 
barriers to stakeholder participation in the delivery of basic education services. It had 15 items, with rotated 
factor loads ranging between 0.34161 and 0.78808, and internal consistency reliability coefficient of 0.8867. 
Sub-scale 4 measured peacefulness and gender sensitivity of learning environments. It had 9 items, with 
rotated factor loads ranging between 0. 31388 and 0.87415, and internal consistency reliability coefficient of 
0.8045. Sub-scale 5 measured pupil participation in co-curricular activities. It had 8 items, with rotated factor 
loads ranging between 0.38487 and 0.77441, and sub-scale internal consistency reliability coefficient of 
0.8126 (see table 1). Inter-factor Pearson correlation coefficients ranged between from 0.3215 to 0.4504 (as 
presented in table 2). The findings show 3-ET to have sound psychometric properties, specifically structure, 
internal consistency reliability and sub-scale distinctiveness as evidenced by the rotated factor loadings (that 
ranged between 0.31 and 0.87), Cronbach’s alpha coefficients (that ranged between 0.80 and 0.88), and 
Pearson correlation coefficients (that ranged between 0.32 and 0.45). The interpretation of [Pearson] 
correlation coefficients in the current study was guided by Cohen’s recommendation. 

From the findings, 3-ET clearly consists of five distinct sub-scales; the first sub-scale measures core 
education support functions and services; sub-scale/factor 2, aesthetics/ efficiency; sub-scale 3, barriers to 
stakeholder participation in basic education services; sub-scale 4 peacefulness and gender sensitivity of 
learning environments and sub-scale 5 pupil participation in co-curricular activities. The inter-factor Pearson 
correlation coefficients were medium, i.e. ranging between 0.3215 and 0.4504. The emerging sub-scales of 3- 
ET are consistent with the core issues addressed by the six components of the REPLICA program namely; 
school-based formal peace education, psychosocial support, community integration, leadership and 
governance, promotion of girls education and use of performing arts in learning. These sub-scales are 
therefore appropriate and can be used to track the hypothesized effects of the REPLICA intervention. 

The procedures used in the current structural validation and reliability testing are standard and 
comparable to those used in other similar studies: particularly factor and variable retention criteria, 
Cronbach’s alpha coefficient threshold and basis for inteipreting Pearson correlation coefficients. This study 
was funded by The Pincer Group International Limited. 

3.3 Limitation 

A key limitation of our study is the fact that the authors did not explore the predictive validity of 3- 
ET; they did not also evaluate its applicability in non-war zones. 
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Table 1. Sub-scale structure with rotated factor loading 


Variable 

Rotated Factor 

Cronbach’s 


Loading 

alpha (a) 

Factor 1 

1 

Existence of 2 rubbish pits 

0.3439 


2 

Indications of regular burning of rubbish 

0.31162 


3 

Clean Compound 

0.33952 


4 

Clean Toilets 

0.31397 


5 

Talking Classes 

0.65607 


6 

Display of time tables in respective classrooms 

0.46688 


7 

Head teacher’s plan for lesson supervision 

0.49105 


8 

Displayed charts in P. 1 in Local Language 

0.66555 


9 

Displayed charts in P. 1 according to themes 

0.63941 


10 

Messages on outer walls of the buildings 

0.50093 

0.8788 

11 

Existence of a Guidance & Counselling room 

0.59719 


12 

Resting place in the G&C room 

0.69309 


13 

Displayed messages in the G&C room 

0.65564 


14 

Record of cases handled under G&C 

0.68471 


15 

Availability of G&C materials 

0.68383 


16 

Indications of use of G&C materials 

0.71354 


17 

A room for girls to uses as a washing facility (bathroom) 

0.75402 


18 

Changing room for girls 

0.68266 


19 

Water for washing hands after visiting the Toilet 

0.32438 


20 

Indications of regular hand washing 

0.4047 



Factor 2 


1 

Properly fixed school Signpost 

0.66269 

2 

Clear words on school signpost 

0.71935 

3 

Schemes of work signed and stamped 

0.79323 

4 

Lesson plans signed and stamped 

0.81262 

5 

Comments of schemes of work and lesson plans 

0.78077 

6 

Availability of files 

0.7869 

7 

Accessibility of files 

0.77999 

8 

Labelling of files 

0.75214 

9 

Current timetable displayed in office 

0.55083 

10 

Display of pupil enrolment figures 

0.42595 

11 

Evidence of pupil enrolment and analysis 

0.45086 

12 

School Mission visibly displayed 

0.37541 

13 

UPE accountability displayed in H/Teacher's office 

0.50955 

14 

Talking office/s 

0.37233 

15 

Record of analysis of teachers' attendance 

0.49533 

16 

Existence of'subject leaders or heads' 

0.53528 

17 

Existence of staff committee or Teacher in charge of sanitation 

0.50177 

18 

List of PTA committee members displayed 

0.55434 

19 

List of SMC committee members displayed 

0.48882 

20 

Minutes of PTA meetings 

0.43054 

21 

Minutes of SMC meetings 

0.52564 
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Factor 3 

1 

Displayed plan for School improvement 


-0.38762 


2 

Record of activities accomplished as per School Improvement plan 


-0.47535 


3 

Existence of a work plan for community involvement 


-0.60638 


4 

Display of the above work plan 


-0.64014 


5 

Record of activities accomplished by the community 


-0.63186 


6 

Work plan for community sensitization activities 


-0.65007 


7 

Display of the above work plan 


-0.75615 


8 

Record of activities accomplished (under sensitization) 


-0.6423 

0.8867 

9 

List of community outreach activities 


-0.6804 


10 

Work plan for community outreach activities 


-0.73919 


11 

Display of the above work plan 


-0.78808 


12 

Record of accomplished community outreach activities 


-0.70579 


13 

A plan for Public Engagement 


-0.45746 


14 

Record of activities accomplished under Public Engagement 


-0.4913 


15 

Record of departmental meetings held 


-0.34161 


Factor 4 

1 

Existence of a Peace Club 


0.87415 


2 

Existence of a Committee for Peace Club 


0.86853 


3 

Activity Plan for Peace Club 


0.74561 


4 

Record of accomplished peace club activities 


0.68533 


5 

Aspects/values of peace incorporated in Schemes of work & lesson Plans 

0.51576 

0.8075 

6 

Existence of a committee for GEM Club 


0.70536 


7 

Activity Plan for GEM Club 


0.76944 


8 

Record of accomplished GEM club activities 


0.52617 


9 

Girls' latrines separate from boys' latrines 


0.31388 


Factor 5 

1 

Functional PALS club 


0.77441 


2 

A teacher in charge of PALS 


0.51245 


3 

A work plan for PALS activities 


0.56905 


4 

5 

Existence of a practice schedule 

A book/file of activities accomplished by PALS club/s 


0.56255 

0.48881 

0.8126 

6 

Availability of accessories for performing arts 


0.68099 


7 

Record of work accomplished under class Supervision by the School Leader 

0.38487 


8 

Compositions and creations on REPLICA 


0.35626 



Table 2. Pair wise correlation of the subscales/ factors 



# Factor 1 Factor 2 

Factor 3 

Factor 4 



Factor 1 0 4452 

Factor 2 0.44 

Factor 3 „ " 0.361 

Factor4 0 4424 °' 3758 

Factor 5 

0.3215 

0.4472 

0.3202 



4. CONCLUSION 

3-ET has acceptable structural validity and internal consistence reliability. It can therefore be used 
to test the effectiveness of the REPLICA intervention. Its applicability in other conflict and non-conflict 
contexts may need specific review. 
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